ASPIRE Outcome Measure Design:
Risk adjustment, provider attribution, and healthcare value
Michael Mathis, MD
Assistant Professor of Anesthesiology
Director, Cardiothoracic Anesthesiology Research
University of Michigan Medical School
http://mpog.org/wp-content/uploads/2018/07/QI-
Story_HenryFord_Hightower.pdf
Tenets of the Ideal Outcome Measure for QI
• Variation in the outcome exists and is measurable
• Variation is attributable to providers (at least in part) participating in the QI collaborative
• All remaining variation in the outcome is explained by factors which can be measured and
risk-adjusted
– Patient characteristics
– Surgical characteristics
• Providers have an opportunity to intervene
– Enables a plan for how to improve
• Outcome has significant healthcare value, as supported by evidence
– Ensures focus on the right measures
How can MPOG achieve these ideals?
• Current outcome measures
– AKI 01: Preventing Acute Kidney Injury
– CARD 02: Avoiding MI (Troponin I <0.6)
– TRAN 02: Post Transfusion Monitoring
Tenet #1: Provider Attribution
• AKI 01: Preventing acute kidney injury
– Who is responsible?
Tenet #1: Provider Attribution
• AKI 01: Preventing acute kidney injury
– Who is responsible?
Provider AKI Mechanism
Anesthesiologist
Hemodynamics, fluid management, diuretics, glycemic control
Tenet #1: Provider Attribution
• AKI 01: Preventing acute kidney injury
– Who is responsible?
Provider AKI Mechanism
Anesthesiologist
Hemodynamics, fluid management, diuretics, glycemic control
Surgeon /
Proceduralist
Direct
injury, physiologic insult, nephrotoxins
Tenet #1: Provider Attribution
• AKI 01: Preventing acute kidney injury
– Who is responsible?
Provider AKI Mechanism
Anesthesiologist
Hemodynamics, fluid management, diuretics, glycemic control
Surgeon /
Proceduralist
Direct
injury, physiologic insult, nephrotoxins
Other
Pre
-existing CKD, comorbid conditions, lifestyle factors
Tenet #1: Provider Attribution
AKI 01: Preventing acute kidney injury
Outcome
Attribution
100%
Case Type
Colectomy AAA Repair
Colonoscopy
Nephrectomy
Anesthesia
Surgeon
Other
Anesthesia
Surgeon
Other
Surgeon
Anesthesia
Endoscopist
Other
Tenet #2: Risk Adjustment
• What is it?
– Method to more accurately assess performance, accounting for baseline risk
• Why do we need it?
– Establishes basis for comparison across providers/institutions with varying baseline risk
– Isolates component of outcome attributable to the anesthesiologist
Tenet #2: Risk Adjustment
• How does it work?
– Compares a provider’s observed performance to what was expected
Tenet #2: Risk Adjustment
• How does it work?
– Compares a provider’s observed performance to what was expected
– Two example providers:
– Who is doing better?
– Incidence of AKI:
– Provider A 20%
– Provider B 3%
Case Type
Performed
# Cases observed
to
have AKI
Provider A
100
AAA repairs
20
Provider B
100 colonoscopies
3
Tenet #2: Risk Adjustment
• Using Risk Adjustment:
Tenet #2: Risk Adjustment
Case Type
Performed
# Cases observed
to
have AKI
Provider A
100
AAA repairs
20
Provider B
100 colonoscopies
3
• Using Risk Adjustment:
– Incidence of AKI:
– Provider A 20%
– Provider B 3%
Tenet #2: Risk Adjustment
Case Type
Performed
# Cases observed
to
have AKI
# Cases
expected
to have AKI
Provider A
100
AAA repairs
20
80
Provider B
100 colonoscopies
3
2
• Using Risk Adjustment:
– Incidence of AKI:
– Provider A 20%
– Provider B 3%
– Comparing provider’s observed performance to what was expected:
Tenet #2: Risk Adjustment
• Using Risk Adjustment:
– Incidence of AKI:
– Provider A 20%
– Provider B 3%
– Comparing provider’s observed performance to what was expected:
– Provider A 0.25
– Provider B 1.50
Case Type
Performed
# Cases observed
to
have AKI
# Cases
expected
to have AKI
Observed /
Expected
(O/E) Ratio
Provider A
100
AAA repairs
20
80
20/80 = 0.25
Provider B
100 colonoscopies
3
2
3/2 = 1.50
Tenet #2: Risk Adjustment
• Analogous to Degree of Difficulty:
• Execution: 9 out of 10
• Degree of Difficulty: x1.0 (cakewalk)
• Execution: ? out of 10
• Degree of Difficulty: x5.0 (extremely challenging)
Colonoscopy
AAA Repair
Tenet #2: Risk Adjustment
• Variables considered in risk adjustment
– Patient characteristics
– Demographics: age, gender, BMI
– ASA status
– Comorbidities: renal insufficiency, HTN, HF, diabetes, CAD, liver disease, etc.
– Labs: hemoglobin, creatinine
– Surgical characteristics
– Procedure type (anesthesia CPT code)
– Emergent / elective
– Center characteristics
– Teaching vs. private hospital
Tenet #2: Risk Adjustment
• Observed/Expected (O/E) Ratio
Provider 1
Provider 2
Provider 3
Provider 4
AKI Rate (%)
Median
0%
100%
O
O
O
O
E
E
E
E
O/E Ratio
High risk patients
High risk procedures
Low risk patients
High risk procedures
High risk patients
Low risk procedures
Low risk patients
Low risk procedures
Tenet #2: Risk Adjustment
• Observed/Expected (O/E) Ratio
Provider 1
High risk patients
High risk procedures
Provider 2
Low risk patients
High risk procedures
Provider 3
High risk patients
Low risk procedures
Provider 4
Low risk patients
Low risk procedures
Median
0%
100%
Provider 1
Provider 2
Provider 3
Provider 4
O/E Ratio
1.0
0
2.0
O
O
O
O
E
E
E
E
O/E
O/E
O/E
O/E
O/E Ratio
AKI Rate (%)
Tenet #3: Opportunity to Intervene
• Necessary for:
– Identifying cases for which provider has a plausible means to improve outcome
• Ideally, tied to process of care measures
– Example: Intraoperative hypotension (process of care) associated with AKI (outcome)
– Creates ability to refine provider attribution, and guide how to improve
Tenet #3: Opportunity to Intervene
• AKI 01: Preventing acute kidney injury
– Exclusions
– Pre-existing renal failure
– ASA 5 & 6
– Surgeries with direct kidney injury (e.g. nephrectomy)
– Short, extremely low risk procedures (ECT, pain, obstetric non-operative)
– Process of care measures
– Low MAP prevention
– (Others)? fluids, glycemic control, transfusion
Tenet #4: Healthcare Value
• Necessary for:
– Allocating anesthesiologist attention & resources to solve the right problems
• Benefits of a quality measure:
– Improves patient care
– Reduces downstream costs associated with complications
• Costs of a quality measure:
– Distracts providers from other unmeasured important care processes
– Produces upstream implementation costs
Tenet #4: Healthcare Value
• Maximizing Anesthesiologist Value
– Outcomes with high healthcare value tend to:
– Be more distant from intraoperative period
– Be less directly associated with intraoperative anesthesiology care
– Outcomes with high anesthesiology provider attribution tend to:
– Be more proximal to the intraoperative period
– Be less directly associated with long-lasting healthcare value
– Anesthesiologist Value = (Overall healthcare value) x (Anesthesiology attribution)
Tenet #4: Healthcare Value
• Outcome time horizon – provider attribution / healthcare value tradeoff
Anesthesiology Attribution
Direct Anesthesiology
Provider Attribution
Short-term
outcomes
Overall
Healthcare Value
Long-term
outcomes
PACU
Outcomes
(Pain / PONV)
Acute Injury
(AKI / MI)
Hospital
Length of Stay /
Readmissions
Long-term
organ injury
(Renal failure, HF)
Mortality
Time Horizon
Moving Forward
• Upcoming outcomes sources
– Surgical registries: NSQIP, MSQC, STS
– PACU/ICU data
– Cost data (MVC)
– Patient reported outcomes
• Transparent & enhanced risk adjustment
– O/E dashboard
– Additional patient & surgical risk factors
– Surgical case type (CPT) prediction
• Linkage of process-of-care outcomes measures
Thank you
